Stereoscopic imaging has been studied for many years in the art. Stereoscopic images allow the perception of depth so to bring viewers with the presence feeling, which has become a hot and promising topic now for home entertainment industry. Stereoscopic imaging is also widely used in applications ranging from military navigation, medical imaging, industrial inspection, to consumer electronics, etc. The principles of stereoscopic imaging and displays are well known. To be specific, each of a left-eye image and a right-eye image which constitute an image pair is taken from slightly different viewpoints, which usually are viewpoints with displacements in a horizontal direction). By separately presenting the left-eye image to the left eye and the right-eye image to the right eye, the human brain will combine them and perceive their disparities as depth.
Conventionally, the image pair can be acquired using capturing devices with two image sensors (e.g. a digital stereo camera). If the captured image pair is properly aligned, comfortable 3D feeling is sensed by the viewers/observers when seen. Unfortunately, the captured image pair is usually not well aligned (for many reasons). For example, alignment of the two image sensors (including optical system) during manufacturing is not perfect. Moreover, the captured images may be distorted due to lens distortion or camera configurations. These can result in horizontal and vertical differences. The difference in the vertical direction and the horizontal direction between two images is known as parallax in general. In order to reduce visual discomfort and eyestrain of the observer and obtain stereoscopic viewing effect, at least vertical parallax must be removed and horizontal parallax needs to be properly adjusted.
In addition, the image pair can be captured using single image sensor devices by sequentially capturing from different viewpoints. More specifically, by capturing an image, moving the camera, and capturing another image, an image pair for stereoscopic viewing can be obtained. There are two advantages of using single sensor devices. One advantage is lower cost compared with two (stereo) sensor devices. Another advantage is the flexibility to change the distance between a left image and a right image according to content of the scene. However, the main problem with this method is the misalignment of the left image and the right image, mainly due to hand movement when using handheld DSC (Digital Still Camera).
Patent Literature 1 (U.S. Pat. No. 6,191,809, issued on Feb. 20, 2001) discloses a method and apparatus for correcting optical misalignment of stereo image capturing system, for example electronic stereo endoscope. The method uses calibration target to compute translation, rotation, and scale, thereby determining calibrated information. Then, the determined calibrated information is used to align image pairs.
Patent Literature 2 (Japanese Unexamined Patent Application Publication No. 2003-244500) discloses an image pickup system for obtaining the stereo image of an object by time sequentially capturing left and right parallax images. In this invention, LCD viewfinder is used for guiding the users to align the two images during capturing.
Patent Literature 3 (U.S. Pat. No. 7,113,632, issued on Sep. 26, 2006) discloses a method for determining rectification transformation of a pair of images.
Patent Literature 4 (U.S. Pat. No. 6,606,404, filed on Jun. 19, 1999) discloses system and method for computing rectifying homographies for stereo vision processing of three dimensional objects. The object of Patent Literature 4 is to transform left and right images such that the epipoles is mapped to the infinity points and the epipolar lines are parallel with the horizontal axis. This method can output images which are suitable for stereoscopic viewing.